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ABSTRACT 


The  objective  is  to  find  out  how  saccadic  eye  movements  (voluntary  jumps  of 
the  eye)  can  be  directed  accurately  to  chosen  targets  within  natural,  patterned  visual 
environments.  An  effective  saccadic  scanning  mechanism  must  consist  of:  (1)  a 
selective  filter  that  determines  which  object  serves  as  the  saccadic  target,  and  (2)  a 
spatial-pooling  mechanism  that  computes  a  single,  precise  saccadic  landing  position. 
Major  findings  were:  (1)  a  single  attentional  filter  serves  both  saccadic  eye 
movements  and  perception;  (2)  attentional  requirements  of  saccades  are  modest;  (3)  it 
is  easier  to  divide  attention  between  two  widely-spaced  targets  than  two  closely-spaced 
targets;  (4)  the  spatial-pooling  process  guides  saccades  to  a  central  landing  position 
within  spatially-extended  targets  with  a  high  degree  of  precision;  (5)  the  central  landing 
position  is  near  the  center  of  gravity;  (6)  the  central  landing  position  and  the  precision 
of  the  saccade  depends  on  target  size  but  not  on  spatial  frequency  content,  contrast,  or 
shape. 
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Objectives 


The  main  objective  of  the  work  is  to  find  out  how  saccadic  eye  movements 
(voluntary  jumps  of  the  eye)  can  be  directed  accurately  to  chosen  targets  within 
natural,  patterned  visual  environments.  The  problem  is  not  one  of  finding  out  which 
targets  are  chosen,  but  rather  to  understand  the  cognitive,  visual  and  motor  mechanisms 
that  carry  out  the  decisions.  These  mechanisms  should  ideally  produce  accurate 
saccadic  patterns  while  at  the  same  time  making  minimal  demands  on  the  cognitive 
resources  otherwise  needed  to  process  the  content  of  the  visual  scene.  Natural 
patterned  visual  scenes  have  two  characteristics  that  present  a  challenge  to  the 
mechanisms  controlling  saccades.  First,  in  natural  scenes  the  targets  for  saccades  are 
spatially-extended  objects.  Second,  in  natural  scenes  the  target  objects  are  surrounded 
by  extraneous  objects  in  the  background.  An  effective  saccadic  scanning  mechanism 
must  consist  of:  (1)  a  selective  filter  that  determines  which  object  or  a  portion  of  the 
visual  scene  serves  as  the  saccadic  target,  and  (2)  a  spatial-pooling  mechanism  that 
operates  on  information  in  a  spatially-extended  target  to  compute  a  single,  precise 
saccadic  landing  position.  Understanding  the  nature  of  the  selective  and  pooling 
processes  that  work  for  human  beings  is  significant  for  designing  visual  displays  that 
support  effective  scanning,  and  for  developing  efficient  procedures  for  the  control  of 
robotic  visual  systems  which,  like  human  beings,  must  guide  sensors  accurately  to 
chosen  targets  within  patterned  scenes. 

Status  of  effort 

We  have  studied  both  the  selective  filter  and  the  spatial  pooling  mechanism  that 
guide  saccades  to  chosen  objects  in  natural  visual  scenes.  Both  processes  work 
together  remarkably  well  to  ensure  efficient  scanning  while  at  the  same  time  imposing 
minimal  cognitive  load.  Major  findings  are:  (1)  a  single  attentional  filter  serves  both 
saccadic  eye  movements  and  perception,  which  means  that  the  line  of  sight  will,  upon 
issuing  a  saccadic  trigger  signal,  automatically  be  directed  toward  targets  of  interest; 

(2)  attentional  requirements  of  saccades  are  modest,  freeing  cognitive  resources  for 
other  concurrent  tasks  and  decisions;  (3)  it  is  easier  to  divide  attention  between  two 
widely-spaced  targets  than  two  closely-spaced  targets;  (4)  a  spatial-pooling  process 
guides  saccades  to  a  central  landing  position  within  spatially-extended  targets  with  a 
surprisingly  high  degree  of  precision;  (5)  the  central  landing  position  is  near  (but  not 
precisely  at)  the  center  of  gravity  of  the  pattern;  (6)  the  locus  of  the  central  landing 
position  and  the  precision  of  the  saccade  depends  on  target  size  but  not  on  other  target 
characteristics,  such  as  spatial  frequency  content,  contrast,  or  shape. 
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Relevance  and  applications 


The  results  show  that  the  oculomotor  and  cognitive  systems  interact  with 
remarkable  efficiency.  For  example,  by  linking  saccadic  target  selection  to  the  same 
attentional  filter  that  serves  perception  it  becomes  unnecessary  for  observers  to  make  a 
separate  decision  about  where  to  aim  the  eye.  The  line  of  sight  will  automatically  go 
to  the  attended  region  after  a  "go"  signal  is  issued  to  trigger  the  movement.  Moreover, 
the  limitations  on  the  attentional  demands  of  saccades  mean  that  considerable  resources 
are  available  for  selection  of  potential  targets  before  the  saccade  is  initiated.  Finally, 
the  finding  of  a  highly-precise  spatial  pooling  of  attended  regions  means  that  observers 
need  only  select  a  target  object;  the  precise  endpoint  of  the  saccade  is  determined 
automatically  by  lower-level  visuomotor  processes  that  do  not  require  deliberate  or 
cognitive  intervention.  Thus,  the  principle,  employed  by  our  visuomotor  system,  of 
using  a  two  stage  process  -  target  object  selection  followed  by  spatial  pooling  -  may 
prove  of  value  in  the  design  of  artificial  visual  systems  that  must,  as  we  do,  aim 
sensors  to  selected  targets  in  large  visual  scenes  with  minimal  drain  on  available 
resources. 


It  is  worthwhile  noting  that  our  information  about  saccadic  scanning  of  natural 
visual  scenes  represents  normal  performance  under  optimal  conditions,  i.e.,  highly- 
visible  displays  inspected  by  highly-motivated  and  attentive  observers.  The 
performance  under  such  conditions  is  remarkably  good  and  is  remarkably  independent 
of  the  spatial  structure  of  the  visual  scene  (two  notable  exceptions:  the  impairment  of 
precision  with  very  large  targets,  see  sections  c  and  d,  below;  and  the  interference 
between  closely-spaced  targets,  summarized  in  section  b,  below).  These  results  serve 
as  an  important  baseline  for  future  work  that  would  evaluate  the  expected  deterioration 
of  performance  when  optimal  viewing  conditions  are  unavailable,  either  because  the 
visibility  of  the  display  or  the  cognitive  state  of  the  observer  is  impaired  or  because 
important  visual  targets  are  crowded  together.  Under  such  conditions,  the  spatial 
layout  of  the  scene  may  become  crucial  in  determining  effective  scanning. 
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Accomplishments  and  new  findings 


(a)  Saccades  and  attention:  These  experiments  explored  the  nature  of  the 
selective  filter  that  guides  saccades  to  chosen  objects.  The  question  was  whether  a 
single  selective  filter  serves  both  perception  and  eye  movements  or  whether  the 
saccadic  system  employed  its  own,  independent  filter.  If  a  shared  selective  filter  is 
involved,  then  it  would  not  be  possible  to  prepare  to  look  at  one  object  while  at  the 
same  time  paying  full  perceptual  attention  to  something  else.  We  devised  several 
"dual-task"  experiments  (concurrent  measures  of  saccades  and  perception)  requiring 
subjects  to  look  at  one  target  while  identifying  another.  Considerable  effort  was 
devoted  to  development  of  dual-task  methods  that  placed  subjects’  strategies  under 
tight  experimental  control.  We  found  that  identification  of  a  perceptual  target  (a 
randomly-chosen  letter)  was  better  at  the  saccadic  goal  than  elsewhere.  Surprisingly,  it 
was  possible  to  accurately  identify  a  letter  at  a  location  other  than  the  saccadic  goal  by 
increasing  saccadic  latency  only  10-30%.  These  results  show  that  saccadic 
programming  does  make  demands  on  perceptual  attention,  but  the  atttentional  demands 
are  modest.  To  explain  the  results,  we  proposed  that  attention  is  not  involved  in 
saccadic  programming  until  shortly  before  saccadic  execution,  and  that  it  is  at  this  time 
that  the  spatial  parameters  of  the  saccade  are  determined.  One  way  to  implement  this 
process  would  by  involving  two  systems,  one  that  determines  the  saccadic  goal 
(controlled  by  means  of  selective  attention)  and  another  that  triggers  the  saccade.  By 
setting  the  saccadic  trigger  to  launch  the  saccade  in  response  to  the  attentional  shift, 
scanning  would  be  both  rapid  and  accurate  while  at  the  same  time  minimizing  the  need 
for  time-consuming  "on-line"  decisions  about  when  and  where  to  aim  the  eye. 

(Kowler,  Anderson,  Dosher,  and  Blaser,  1995). 

(b)  Attentional  interference:  We  performed  a  perceptual  version  of  the 
experiment  described  above.  Instead  of  looking  at  one  target  and  trying  to  identify 
another,  we  asked  subjects  to  identify  two  targets  in  a  circular  display  (rad  4  deg)  of 
24  letters  (Figure  1A).  The  objective  was  to  measure  the  size  of  an  ’attentional 
window’,  i.e.,  the  region  over  which  information  can  be  acquired  in  a  single  attentional 
glimpse.  The  expected  outcome  was  that  it  should  be  easy  to  identify  a  pair  of  letters 
so  close  to  one  another  that  they  would  fall  within  the  same  attentional  window.  By 
finding  how  performance  declines  with  increasing  separation,  it  would  be  possible  to 
estimate  the  size  of  the  window. 

The  experimental  sequence  (Fig.  IB)  consisted  of  a  pre-trial  frame,  pre-mask 
(500  msec),  critical  display  frame  containing  the  target  letter  (100,  200  or  300  msec) 
and  a  post-mask  (500  msec).  The  locations  of  the  target  letters  were  indicated  either 
by  color  cues  present  in  (1)  all  frames  or  (2)  the  pre-trial  frame  only,  or  by  numeral 
cues  present  in  the  pre-trial  frame.  Different  types  of  cues  were  tested  to  ensure  that 
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the  results  were  not  due  to  sensory  attributes  of  the  display  but  to  the  attentional 
requirements. 

Contrary  to  expectations,  target  identification  improved  with  increasing  target 
separation.  Figure  2  shows  that  the  improvement  held  for  all  3  subjects  and  all  3 
critical  frame  durations.  Further  analyses  showed  that  there  were  two  types  of  errors 
made  at  close  spatial  separations,  namely,  identifying  a  neighboring  letter  instead  of 
the  target  and  (2)  failure  to  identify  either  the  target  or  the  neighbor  accurately.  Errors 
were  more  frequent  when  the  pair  of  targets  fell  within  a  single  hemisphere. 

These  novel  results  show  that  there  is  a  limit  on  how  much  information  can  be 
processed  in  any  local  region  during  a  single  attentional  glimpse,  i.e.,  processing  at  one 
location  is  enhanced  by  ’borrowing’  resources  from  nearby  regions.  At  a  neural  level, 
such  adjustments  in  processing  resources  may  be  caused  by  changes  in  neural  firing 
rates,  or  changes  in  receptive  field  tuning.  Optimal  visual  performance  would  be 
achieved  by  designing  displays  in  which  important  features  are  widely-spaced.  (Bahcall 
and  Kowler,  1995). 

(c)  Saccadic  accuracy  and  precision:  These  experiments  investigated  the 
pooling  process  that  brings  the  line  of  sight  to  spatially-extended  targets.  Unlike  the 
studies  on  spatial  filtering  described  above,  the  stimulus  was  a  single  target  and  the 
instruction  was  to  look  at  the  target  as  a  whole,  rather  than  to  aim  for  a  specific  place 
within  it.  The  main  result  was  that  saccadic  accuracy  and  precision  were  quite  good. 
With  small,  single  point  targets  average  landing  positions  missed  the  target  by  only  1% 
of  eccentricity  with  standard  deviations  of  saccadic  landing  position  only  6%  of  the 
eccentricity.  This  level  of  precision  is  comparable  to  that  of  relative  perceptual 
localization.  This  high  level  of  performance  was  maintained  even  for  very  large 
targets  (up  to  3  deg  diam  at  an  eccentricity  of  4  deg).  Precision  was  impaired  for 
larger  targets.  The  independence  from  target  size  suggests  that  a  pooling  mechanism 
operates  on  spatially-extended  targets  to  determine  a  precise  landing  position  without 
adding  much  noise  to  the  computation.  The  landing  position  is  sufficiently  precise  that 
it  is  now  possible  to  do  experiments  the  manipulate  target  configuration  so  as  to 
understand  how  the  pooling  process  works.  (Kowler  and  Blaser,  1995). 

(di  Saccades  to  dot  clusters:  The  pooling  process  that  guides  saccades  to 
spatially-extended  targets  was  investigated  by  studying  saccades  made  to  4  deg  diam 
clusters  of  19  random  dots  (Fig.  3).  Random  dot  clusters  were  studied  in  order  for  us 
to  study  sensory  aspects  of  the  spatial  pooling  process  with  minimal  interference  from 
deliberate  strategies  of  saccadic  guidance.  Any  such  strategies  would  not  be  too 
effective  with  random  dot  targets  because  the  location  of  the  center  of  gravity  of  the 
patterns  was  not  obvious  in  eccentric  vision,  nor  could  it  be  inferred  from  previously 
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presented  targets.  Eye  movements  were  recorded  via  a  Dual  Purkinje  Eyetracker 
(noise  <  1’)  while  subjects  were  instructed  to  make  a  single  saccade,  as  accurately  as 
possible,  to  the  dot  cluster.  Instructions  were  to  aim  at  the  cluster  as  a  whole  rather 
than  at  a  specific  place  within  it. 

The  main  finding  was  that  saccades  landed  at  reliable  places  within  the  pattern. 
The  landing  position  was  near  but  not  precisely  at  the  center-of-gravity.  Slopes  of  the 
functions  relating  saccadic  landing  position  to  center-of-gravity  (Fig.  4a, b)  were  less 
than  1  and  the  variability  of  saccades  (after  center-of-gravity  was  taken  into  account) 
was  only  10%  of  eccentricity  (larger  than  the  variability  of  saccades  to  single  point 
targets).  Correlations  of  saccadic  landing  position  with  the  presence  of  a  dot  at  any 
display  location  showed  that  dots  from  the  entire  display  were  taken  into  account  in 
determining  the  landing  position  of  the  saccade.  Assigning  differential  weights  to 
different  portions  of  the  cluster  did  not  improve  the  prediction  of  saccadic  landing 
position,  indicating  dots  all  contributed  regardless  of  whether  they  were  located  near 
the  center  or  the  boundary,  and  regardless  of  their  eccentricity.  Varying  dot  intensity 
over  a  log  unit  did  matter,  with  the  bright  dots  influencing  landing  position  about  twice 
as  much  as  the  dimmer  dots. 

These  results  show  that  the  spatial  pooling  process  computes  with  unexpectedly 
high  precision  a  central  landing  position  within  unstructured  spatially-extended  targets. 
The  observed  departures  of  this  landing  position  from  the  center  of  gravity  were  not 
due  differential  weighting  of  dot  locations.  We  suspect  that  such  departures  were 
caused  by  processes  subsequent  to  the  center  of  gravity  computation  involved  with 
transforming  the  visual  error  signal  into  the  saccadic  command.  The  results  also  show 
that  the  pooling  process  was  sensitive  to  local  characteristics  of  the  pattern,  thus  ruling 
out  models  based  on  low-pass  filtering.  It  is  more  likely  that  pooling  operates  by 
averaging  the  local  signs  of  units  sensitive  to  different  portions  of  the  pattern,  with  the 
contribution  of  each  unit  determined  by  the  number  and  intensity  of  the  dots  in  its 
receptive  field.  One  would  expect  that  increasing  the  number  of  such  units  should 
increase  the  spatial  precision  of  the  saccade,  but  this  did  not  happen:  saccades  to  4 
deg  diameter  targets  are  less  precise  than  saccades  to  single  points.  A  good 
explanation  for  this  loss  of  precision  with  larger  targets  is  that  the  receptive  field  size 
(and,  hence,  the  spatial  precision)  of  the  units  involved  becomes  larger  when  the 
stimulus  is  larger.  Thus,  it  may  not  be  possible  for  the  visual  system  to  spatially  pool 
the  output  of  units  with  small  receptive  fields  over  too  wide  an  area.  (McGowan, 
Kowler  and  Sharma). 

(e)  Saccades  to  spatially-filtered  targets:  The  spatial  pooling  process  is  being 
investigated  by  studying  the  precision  of  saccades  directed  to  Gabor  patches  (sinusoidal 
gratings  whose  contrast  is  modulated  by  a  Gaussian  envelope).  The  main  objective  is 
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to  find  out  whether  saccades  use  the  same  set  of  spatial  filters  as  the  perceptual 
system,  or  whether  some  spatial  frequencies,  although  visible,  constitute  ineffective 
(i.e.,  noisy)  targets  for  saccades.  Results  show  that  saccades  are  insensitive  to  spatial 
frequency  and  to  contrast,  once  stimuli  reach  threshold.  (McGowan,  Kowler,  Bergen 
and  Lubin.) 

(f)  Saccades  to  targets  of  different  shapes:  Theories  of  form  perception  have 
given  a  prominent  role  to  the  "medial  axis"  of  forms  (Blum,  1974)  and  recent 
psychophysical  data  (Kovacs  and  Julesz,  1994)  suggest  that  the  medial  axis  may  indeed 
be  computed  by  units  that  pool  information  across  the  contour.  We  studied  saccades  to 
forms  in  which  the  medial  axis  and  center  of  gravity  were  in  different  locations. 
Saccadic  landing  position  favored  the  medial  axis  and  saccadic  precision  was 
independent  of  the  shape.  Follow  up  experiments  will  study  landing  positions  to 
shapes  during  performance  of  a  "real"  tasks  (e.g.,  memorize  the  shapes;  compare  the 
shapes  to  find  out  which  one  is  different).  We  want  to  find  out  whether  the 
demonstrated  capacity  of  the  saccadic  system  for  precise  localization  is  used  during 
these  tasks,  or  whether  it  sets  an  upper  bound  on  performance  which  far  exceeds  the 
demands  made  by  even  difficult  perceptual  tasks.  The  latter  outcome  indicates  that  the 
perceptual  and  cognitive  systems  would  have  considerable  flexibility  in  determining 
landing  position  unimpaired  by  sensorimotor  noise.  (Melcher  and  Kowler). 
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